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Two objects are independent if they do not affect each other. Independence is well- 
understood in classical information theory, but less in algorithmic information theory. 
\ Working in the framework of algorithmic information theory, the paper proposes two 

r T ^ types of independence for arbitrary infinite binary sequences and studies their proper- 

ties. Our two proposed notions of independence have some of the intuitive properties 
that one naturally expects. For example, for every sequence x, the set of sequences 
that are independent (in the weaker of the two senses) with x has measure one. For 
both notions of independence we investigate to what extent pairs of independent se- 
quences, can be effectively constructed via Turing reductions (from one or more input 
sequences). In this respect, we prove several impossibility results. For example, it 
is shown that there is no effective way of producing from an arbitrary sequence with 
positive constructive Hausdorff dimension two sequences that are independent (even 
in the weaker type of independence) and have super-logarithmic complexity. Finally, 
■ a few conjectures and open questions are discussed. 

OO 

1 1 Introduction 

<N 

Intuitively, two objects are independent if they do not affect each other. The concept is 
well-understood in classical information theory. There, the objects are random variables, 
the information in a random variable is its Shannon entropy, and two random variables X 
and Y are declared to be independent if the information in the join (X, Y) is equal to the 
^ ■ sum of the information in X and the information in Y. This is equivalent to saying that the 

information in X conditioned by Y is equal to the information in X, with the interpretation 
that, on average, knowing a particular value of Y does not affect the information in X. 

The notion of independence has been defined in algorithmic information theory as well 
for finite strings [Cha82]. The approach is very similar. This time the information in a 
string x is the complexity (plain or prefix-free) of x, and two strings x and y are independent 
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if the information in the join string (x, y) is equal to the sum of the information in x and 
the information in y, up to logarithmic (or, in some cases, constant) precision. 

The case of infinite sequences (in short, sequences) has been less studied. An inspection 
of the literature reveals that for this setting, independence has been considered to be 
synonymous with pairwise relative randomness, i.e., two sequences x and y are said to be 
independent if they are (Martin-L6f ) random relative to each other (see [vL90t IDH] ) . The 
effect of this approach is that the notion of independence is confined to the situation where 
the sequences are random. 

The main objective of this paper is to put forward a concept of independence that 
applies to all sequences. One can envision various ways for doing this. One possibility 
is to use Levin's notion of mutual information for sequences I.evH ij (see also the survey 
paper [GV04]) and declare two sequences to be independent if their mutual information 
is small. If one pursues this direction, the main issue is to determine the right definition 
for "small." We take another approach, which consists in extending in the natural way 
the notion of independence from finite strings to sequences. This leads us to two con- 
cepts: independence and finitary-independence. We say that (1) two sequences x and y 
are independent if, for all n, the complexity of x\n (the prefix of x of length n) and the 
complexity of x\n relativized with y are within O(logn) (and the same relation holds if 
we swap the roles of x and y), and (2) two sequences x and y are finitary-independent if, 
for all n and m, the complexity of x\n and the complexity of x\n given y\m are within 
0(logn + logm) (and the same relation holds if we swap the roles of x and y). We have 
settled for the additive logarithmical term of precision (rather than some higher accuracy) 
since this provides robustness with respect to the type of complexity (plain or prefix-free) 
and other technical advantages. 

We establish a series of basic facts regarding the proposed notions of independence. We 
show that independence is strictly stronger than finitary-independence. The two notions of 
independence apply to a larger category of sequences than the family of random sequences, 
as intended. However, they are too rough for being relevant for computable sequences. It 
is not hard to see that a computable sequence x is independent with any other sequence y, 
simply because the information in x can be obtained directly. In fact, this type of trivial 
independence holds for a larger type of sequences, namely for any .ff -trivial sequence, and 
trivial finitary-independence holds for any sequence x whose prefixes have logarithmic com- 
plexity. It seems that for this type of sequences (computable or with very low complexity) 
a more refined definition of independence is needed (perhaps, based on resource-bounded 
complexity). We show that the two proposed notions of independence have some of the 
intuitive properties that one naturally expects. For example, for every sequence x, the set 
of sequences that are finitary-independent with x has measure one. The same issue for 
independence remains open. 

We next investigate to what extent pairs of independent, or finitary-independent se- 
quences, can be effectively constructed via Turing reductions. For example, is there a 
Turing reduction / that given oracle access to an arbitrary sequence x produces a sequence 
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that is finitary-independent with x? Clearly, if we allow the output of / to be a computable 
sequence, then the answer is positive by the type of trivial finitary-independence that we 
have noted above. We show that if we insist that the output of / has super-logarithmic 
complexity whenever x has positive constructive Hausdorff dimension, then the answer is 
negative. In the same vein, it is shown that there is no effective way of producing from an 
arbitrary sequence x with positive constructive Hausdorff dimension two sequences that 
are finitary-independent and have super-logarithmic complexity. 

Similar questions are considered for the situation when we are given two (unitary-) 
independent sequences. It is shown that there are independent sequences x and y and a 
Turing reduction g such that x and g(y) are not independent. This appears to be a bad 
artifact of the notion of independence proposed in this paper. We consider that this is the 
only counter-intuitive effect of our definitions. We do not know if a similar phenomenon 
holds for finitary-independence. On the other hand, for any independent sequences x and 
y and for any Turing reduction g, x and g(y) are finitary-independent. 

We also raise the question on whether given as input several (unitary-) independent 
sequences x and y it is possible to effectively build a new sequence that is (unitary-) 
independent (not in the trivial way) with each sequence in the input. It is observed that 
the answer is positive if the sequences in the input are random, but for other types of 
sequences the question remains open. The same issue can be raised regarding finite strings 
and for this case a positive answer is obtained. Namely, it is shown that given three 
independent finite strings x, y and z with linear complexity, one can effectively construct a 
new string that is independent with each of x, y and z, and has high complexity and length 
a constant fraction of the length of x, y and z. 

1.1 Preliminaries 

N denotes the set of non-negative integers; the size of a finite set A is denoted \\A\\. 
Unless stated otherwise, all numbers are in N and all logs are in base 2. We work 
over the binary alphabet {0,1}. A string is an element of {0,1}* and a sequence is 
an element of {0,1}°°. If x is a string, |x| denotes its length; xy denotes the concate- 
nation of the strings x and y. If x is a string or a sequence, x(i) denotes the i-th 
bit of x and x\n is the substring x{l)x(2) ■ ■ ■ x(n). For two sequences x and y, x © y 
denotes the sequence x(l)y(l)x(2)y(2)x(3)y(3) • • • and x XOR y denotes the sequence 
(x(l) XOR y(l))(x(2) XOR y(2))(x(3) XOR y(3)) • • • , where (x(i) XOR y(i)) is the sum 
modulo 2 of the bits x{i) and y(i). We identify a sequence x with the set {n € N | x(n) = 1}. 
We say that a sequence x is computable (computably enumerable, or c.e.) if the corre- 
sponding set is computable (respectively, computably enumerable, or c.e.). If x is c.e., 
then for every s £ N, x s is the sequence corresponding to the set of elements enumerated 
within s steps by some machine M that enumerates x (the machine M is given in the 
context). We also identify a sequence x with the real number in the interval [0, 1] whose 
binary writing is 0.x(l)x(2) • • • . A sequence x is said to be left c.e. if the corresponding 
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real number x is the limit of a computable increasing sequence of rational numbers. The 
plain and the prefix-free complexities of a string are defined in the standard way; however 
we need to provide a few details regarding the computational models. The machines that 
we consider process information given in three forms: (1) the input, (2) the oracle set, (3) 
the conditional string. Correspondingly, a universal machine has 3 tapes: 

• one tape for the input and work, 

• one tape for storing the conditional string, 

• one tape (called the oracle-query tape) for formulating queries to the oracle. 

The oracle is a string or a sequence. If the machine enters the query state and the value 
written in binary on the oracle-query tape is n, then the machine gets the n-th bit in the 
oracle, or if n is larger than the length of the oracle, the machine enters an infinite loop. 

We fix such a universal machine U . The notation U w (u \ v) means that the input is 
u, the conditional string v and the oracle is given by w, which is a string or a sequence. 
The plain complexity of a string x given the oracle w and the conditional string v is 
C w (x | v) = min{|ii| | U w (u \ v) = x}. There exists a constant c such that for every x,v 
and w C w (x \ v) < \x\ + c. 

A machine is prefix- free (self-delimiting) if its domain is a prefix- free set. There exist 
universal prefix- free machines; we fix such a machine U; the prefix- free complexity of a 
string x given the oracle w and the conditional string v is H w (x \ v) = min{|u| | U w (u \ 
v) = x}. 

In case w or v are the empty strings, we omit them in C(-) and H(-). Throughout 
this paper we use the O(-) notation to hide constants that depend only on the choice 
of the universal machine underlying the definitions of the complexities C and H. Since 
the prefix- free universal machine is a particular type of machine, it follows that C w (x \ 
v) < H w (x | v) + 0(1), for every x,v and w. The reverse inequality between C(-) and 
H(-) also holds true, within an additive logarithmic term, and is obtained as follows. 
For example, a string x = x{l)x(2) ■ ■ ■ x(n) can be coded in a self-delimiting way by 
x I— > code(x) = 11 • • ■ 1 0bin(n)x(l)x(2) • • • x(n), where bin(n) is the binary representation 

|bin(n)| 

of n € N. Note that \code(x)\ = \x\ + 2 log |x| + 0(1). This implies that for every x,v, and 
w, 

C w {x | v) > H w {x | v) - 21og |x| - 0(1). (1) 
The following inequalities hold for all strings x and y: 

Cy(x)<C(x\y) + 2log\y\ + 0(l), (2) 
\C(xy) - (C(x\y) + C(y))\ < 0(logO(x) + logO(y)). (3) 
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The first inequality is easy to derive directly; the second one is called the Symmetry of 
Information Theorem, see [ZL70J. 

There are various equivalent definitions for (algorithmic) random sequences as defined 
by Martin-L6f [ML66] (see |C02j ). In what follows we will use the (weak) complexity- 
theoretic one [Cha75| using the prefix-free complexity: A sequence x is Martin-L6f random 
(in short, random) if there is a constant c such that for every n, H(x\n) > n — c. The set 
of random sequences has constructive (Lebesgue) measure one [ML66]. 

The sequence x is random relative to the sequence y if there is a constant c such that for 
every n, H y {x\n) > n — c. Note that if x is random, then for every n, C{x\n) > n — 21ogn — 
0(1) (by inequality ([T])). A similar inequality also holds for the relativized complexities, 
i.e. for all x that are random relative to y and for all n, C y {x\n) > n — 21ogn — 0(1). 
These results will be repeatedly used throughout the paper. 

In [vL90] van Lambalgen proves that x © y is random iff x is random and y is random 
relative to x. This implies that if x is random and y is random relative to x then x is 
random relative to y. 

The constructive Hausdorff dimension of a sequence x — which is the di- 
rect effectivization of "classical Hausdorff dimension" — defined by dim(x) = 
lim inf^^oo C(x\n)/n (= liminf^^oo H(x\n)/n), measures intermediate levels of random- 
ness (see |Rya84j ISta93| ITad02l |May02j ILut031 IRei04l ISta051 ICST061 iDHNTOf^ . 

A Turing reduction / is an oracle Turing machine; f(x) is the language computed by / 
with oracle x, assuming that / halts on all inputs when working with oracle x (otherwise 
we say that f(x) does not exist). In other words, if n € f(x) then the machine / on input 
n and with oracle x halts and outputs 1 and if n $ f(x) then the machine / on input n 
and with oracle x halts and outputs 0. The function use is defined as follows: use x j{n) 
is the index of the rightmost position on the tape of / accessed during the computation 
of / with oracle x on input n. The Turing reduction / is a wtt-reduction if there is a 
computable function q such that use x Jn) < q(n), for all n. The Turing reduction / is a 
truth-table reduction if / halts on all inputs for every oracle. A truth-table reduction is a 
wtt-reduction. 



2 Defining independence 

The basic idea is to declare that two objects are independent if none of them contains 
significant information about the other one. Thus, if in some formalization, I{x) denotes 
the information in x and I{x \ y) denotes the information in x given y, x and y are 
independent if I(x) — I(x | y) and I(y) — I(y \ x) are both small. In this paper we work 
in the framework of algorithmic information theory. In this setting, in case x is a string, 
I(x) is the complexity of x (where for the "complexity of x" there are several possibilities, 
the main ones being the plain complexity or the prefix- free complexity). 

The independence of strings was studied in [Cha82]: two strings are independent if 
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I(xy) ~ I (a;) + I(y). This approach motivates our Definition 12.11 and Definition 12.21 

In case x is an infinite sequence, the information in x is characterized by the sequence 
( J(x |n)) n £N °f information in the initial segments of x. In the infinite case, for the infor- 
mation upon which we condition (e.g., the y in I{x \ y)), there are two possibilities: either 
the entire sequence is available in the form of an oracle, or we consider initial segments of 
it. Accordingly, we propose two notions of independence. 

Definition 2.1 (The "integral" type of independence) Two sequences x and y are 
independent if C x {y\n) > C{y\n) — O(logra) and C y {x\n) > C{x\n) — O(logn). 

Definition 2.2 (The finitary type of independence) Two sequences x, y are finitary- 
independent if for all natural numbers n and m, 

C(x\n y\m) > C{x\n) + C(y\m) — 0(log(n) + log(m)). 

Remark 1 We will show in Proposition 12.41 that the inequality in Definition 12.21 is equiv- 
alent to saying that for all n and m, C(x\n \ y\m) > C(x\n) — 0(logn + log to), which 
is the finite analogue of the property in Definition 12.11 and is in line with our discussion 
above. 

Remark 2 If x and y are independent, then they are also finitary-independent (Proposi- 
tion [275]) . The converse is not true (Corollary 14. 13j) . 

Remark 3 The proposed definitions use the plain complexity C(-), but we could have 
used the prefix-free complexity as well, because the two types of complexity are within 
an additive logarithmic term. Also, in Definition 12.21 (and throughout this paper), we use 
concatenation to represent the joining of two strings. However, since any reasonable pairing 
function (x,y) satisfies | \(x,y)\ — \xy\ \ < 0(log|x| +log|y|), it follows that |C(< x,y > 
) — C{xy)\ < 0(log \x\ + log \y\), and thus any reasonable pairing function could have been 
used instead. 

Remark 4 A debatable issue is the subtraction of the logarithmic term. Indeed, there 
are other natural possibilities. We argue that our choice has certain advantages over other 
possibilities that come to mind. 

Let us focus on the definition of finitary- independence. We want C(x\n y\m) > 
C{x\n) + C{y\n) — 0(f(x) + f(y)), for all n, to, where / should be some "small" function. 
We would like the following two properties to hold: 

(A) the sequences x and y are finitary-independent iff C(x\n \ y\m) > C{x\n) — 
0(f{x\n) + f(y\m)), for all n and to, 

(B) if x is "somewhat" random and y = W , then x and y are finitary-independent. 
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Other natural possibilities for the definition could be: 

(i) if f(x) = C(\x\), the definition of finitary independence-(i) would now be: 

C{x\n y\m) > C{x\n) + C{y\m) - 0{C(n) + C(m)), 

or 

(ii) if f{x) = logC(x), the definition of finitary-independence-(ii) would now be: 

C(x\n y\m) > C{x\n) + C{y\m) — 0(logC(x\n) + log C(y \m) ) . 

If sequences x and y satisfy (i), or (ii), then they also satisfy Definition 12.21 
Variant (i) implies (B), but not (A) (for example, consider sequences x and y with 
C(n) << logC(x|"n) and C{m) « log C(y\m), for infinitely many n and m). Variant 
(ii) implies (A), but does not imply (B) (for example if for infinitely many n, C{x\n) = 
0(log 3 n); take such a value n, let p be a shortest description of x\n, and let m be the 
integer whose binary representation is lp. Then x\n and O^fm, do not satisfy (B)). The 
proposed definition implies both (A) and (B). 

Another advantage is the robustness properties from Remark [3j 

Remark 5 If the sequence x is computable, then x is independent with every sequence y. 
In fact a stronger fact holds. A sequence is called //-trivial if, for all n, H{x\n) < H(n) + 
O(l). This is a notion that has been intensively studied recently (see [DHNT06 ). Clearly 
every computable sequence is //-trivial, but the converse does not hold |Zam9CH ISol75| . If 
x is //-trivial, then it is independent with every sequence y. Indeed, H v (x\n) > H(x\n) — 
O(logra), because H{x\n) < H(n)+0(1) < logn+0(l), and H x {y \n) > H(y\n) — O(logn), 
because, in fact, H x (y\n) and H(y\n) are within a constant of each other [Nie05j . The 
same inequalities hold if we use the C(-) complexity (see Remark [3]). 

For the case of finitary-independence, a similar phenomenon holds for a (seemingly) 
even larger class. 

Definition 2.3 A sequence x is called C-logarithmic if C{x\n) = O(logn). 

It can be shown (for example using Proposition 12.41 ( a )) that if x is C-logarithmic, then it 
is finitary-independent with every sequence y. 

Note that every sequence x that is the characteristic sequence of a c.e. set is C- 
logarithmic. This follows from the observation that, for every n, the initial segment x\n 
can be constructed given the number of l's in x\n (an information which can be written 
with logn bits) and the finite description of the enumerator of the set represented by x. If 
a sequence is //-trivial then it is C-logarithmic, but the converse probably does not hold. 

In brief, the notions of independence and finitary-independence are relevant for strings 
having complexity above that of //-trivial sequences, respectively C-logarithmic sequences. 
The cases of independent (finitary-independent) pairs (x,y), where at least one of x and y 
is //-trivial (respectively, C-logarithmic) will be referred to as trivial independence. 
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Remark 6 Some desirable properties of the independence relation are: 
PI. Symmetry: x is independent with y iff y is independent with x. 
P2. Robustness under type of complexity (plain or prefix-free). 

P3. If / is a Turing reduction, except for some special cases, x and f(x) are dependent 
("independence cannot be created"). 

P4. For every x, the set of sequences that are dependent with x is small (i.e., it has 
measure zero). 

Clearly both the independence and the finitary- independence relations satisfy PI. They 
also satisfy P2, as we noted in Remark 

It is easy to see that the independence relation satisfies P3, whenever we require that 
the initial segments of x and f(x) have plain complexity a;(logn) (because C x (f(x)\n) = 
O(logn), while C(f(x)\n) = w(logn)). We shall see that the finitary-independence rela- 
tion satisfies P3 under some stronger assumptions for / and f(x) (see Section [4.11 and in 
particular Theorem I4.8p . 

We do not know whether the independence relation satisfies P4. Theorem 13.31 shows 
that the finitary-independence relation satisfies P4. 

2.1 Properties of independent and finitary-independent sequences 

The following simple properties of finitary-independent sequences are technically useful in 
some of the next proofs. 

Proposition 2.4 (a) Two sequences x and y are finitary-independent 44> for all n and 
m, C(x\n | y\m) > C(x\n) — 0(logn + logm). 

(b) Two sequences x and y are finitary-independent if and only if for alln, C (x \n y \n) > 
C{x\n) + C{y\n) - 0(log(n)). 

(c) Two sequences x and y are finitary-independent if and only if for all n, C(x\n \ 
y\n) > C(x\n) — 0(log(n)). 

(d) If x and y are not finitary-independent, then for every constant c there are infinitely 
many n such that C(x\n y\n) < C{x\n) + C(y\n) — clogn. 

(e) If x and y are not finitary-independent, then for every constant c there are infinitely 
many n such that C(x\n \ y\n) < C(x\n) — clogn. 

Proof. We use the following inequalities which hold for every strings x and y (they 
follow from the Symmetry of Information Equation ([3]) ) : 

C(xy) > C(x) + C(y | x) - 0(log [x| + log \y\), (4) 
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and 

C(xy) < C{x) + C(y \ x) + 0(Iog \x\ + log \y\). (5) 

(a) "=>" 

C{x\n\y\m) > C(x \n y\ m) — C(y\m) — O (log n + log m) (by©) 

> C(a? |~n) + C(y \m) — C(y \m) — O (log n + log m) (by independence) 
= C{x\n) — Oilogn + logm). 

C(x\ny\m) > C(y\m) + C{x\n \ y \m) — 0(logn + logm) (by (JH)) 

> C(y\m) + C{x\n) — 0(logn + logm) (by hypothesis). 

(b) "=>■" Take n = m. 

"<^=" Suppose n > m (the other case can be handled similarly). 

C{x\ny\m) > C{y \m) + C(x\n \ y\m) — 0(log(n) + log(m)) (by @) 

> C(y\m) + C{x\n \ y\n) — 0(log(n) + log(m)) 

> C(y\m) + C(x\n) - 0(log(n) +log(m)) (by (a)). 

(c) This follows from (b) with a similar proof as for (a). 

(d) Suppose that for some constant c the inequality holds only for finitely many n. 
Then one can choose a constant c' > c for which the opposite inequality holds for every n, 
which by (b) would imply the nnitary-independence of x and y. 

(e) Follows from (c), in a similar way as (d) follows from (b). □ 

Proposition 2.5 // the sequences x and y are independent, then they are also finitary- 
independent. 

Proof. Suppose x and y are not finitary- independent. By Proposition 12.41 (e), for every 
constant c there are infinitely many n such that C(x\n \ y\n) < C(x\n) — c ■ logn. Taking 
into account inequality ([2]), we obtain C y (x \n) < C(x \n) — (c — 3) log n, for infinitely many 
n, which contradicts that x and y are independent. □ 

Proposition 2.6 //dim(x) = a and(x,y) are finitary-independent, then dim(x XOR y) > 
a. 

Proof. Note that C{x\n \ y\n) < C{(x XOR y)\n) + 0(1), for all n (this holds for all 
sequences x and y). Suppose there exists e > such that dim(x XOR y) < a — e. It follows 
that, for infinitely many n, C({x XOR y)\n) < (a — e)n. Then 

C{x\n\y\n) < C((x XOR y) \n) + O(l) 

< (er — e)n + O(l) for infinitely many n. 
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By the finitary-independence of (x,y), C{x\n) < C(x\n \ y\n) + O(logra) < (a — e/2)n + 
O(l), i.o. n, which contradicts the fact that dim(x) = a. □ 

Proposition 2.7 (a) If x is random and (x,y) are finitary-independent, then 
(y, x XOR y) are finitary-independent. 

(b) If x is random and (x,y) are independent, then (y,x XOR y) are independent. 

Proof. We prove (a) ((b) is similar). Suppose that y and x XOR y are not finitary- 
independent. Then for every constant c, there are infinitely many n, such that 
C((x XOR y)\n \ y\n) < C((x XOR y)\n) — clogn. Note that if a program can pro- 
duce (x XOR y) \n given y \n, then by doing an extra bitwise XOR with y \n it will produce 
x\n. Thus, C{x\n \ y\n) < C((x XOR y) \n \ y\n) + O(l) for all n. Combining with the 
first inequality, for every constant c and for infinitely many n we have: 

C{x\n | y\n) < C((x XOR y) \n) - clog n + O(l) 

< n — clog n + 0(1) 

< C(x \n) + 2 log n — c log n + O(l) 
= C(x \n) - (c - 2) log n + 0(1). 

This contradicts the fact that x and y are finitary-independent. □ 

Proposition 2.8 There are sequences x,y, and z such that (x,y) are independent, (x,z) 
are independent, but (x, y © z) are not finitary-independent. 

Proof. Take y and z two sequences that are random relative to each other, and let x = 
y XOR z. Then (x, y) are independent, and (x, z) are independent, by Proposition 12.71 On 
the other hand note that dim(y XOR z) = 1 (by Proposition ^. 6|) and C((y XOR z) \n\ {y@ 
z) \2n) < 0(1). Consequently, for every constant c and for almost every n, C((y XOR z) \n \ 
(y © z)\2n) < C({y XOR z)\n) - c(logn + log2n), and thus, (y XOR z,y © z) are not 
finitary-independent. □ 

In Remark [5l we have listed several types of sequences that are independent or finitary- 
independent with any other sequence. The next result goes in the opposite direction: it 
exhibits a pair of sequences that can not be finitary-independent (and thus not indepen- 
dent). 

Proposition 2.9 |Ste07| If x and y are left c.e. sequences, dim(x) > ; and dim(y) > 0, 
then x and y are not finitary-independent. 

Proof. For each n, let cm x (n) = min{s [ x s \n = x\n} and cm y (n) = min{s \y s \n = y\n} 
(the convergence moduli of x and, respectively, y). Without loss of generality we can assume 
that cm x (n) > cm y (n), for infinitely many n. For each n satisfying the inequality, y\n can 
be computed from x\n as follows. First compute s = cm^n) (which can be done because 
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x\n is known) and output y s \n. Consequently, for infinitely many n, C(y\n \ x\n) < 0(1). 
On the other hand, since dim(y) > 0, there exists a constant c such that C(y\n) > c ■ n, 
for almost every n. Consequently, x and y are not finitary- independent. □ 

3 Examples of independent and finitary-independent se- 
quences 

We give examples of pairs of sequences that are independent or finitary-independent (other 
than the trivial examples from Remark [5]). 

Theorem 3.1 Let x be a random sequence and let y be a sequence that is random relative 
to x. Then x and y are independent. 

Proof. Since y is random relative to x, for all n, C x {y\n) > n — 21ogn — O(l) > C(y\n) — 
21ogn — 0(1). The van Lambalgen Theorem [vL90] implies that x is random relative to y 
as well. Therefore, in the same way, C y (x\n) > n — 21ogn — 0(1) > C(x\n) — O(logn). □ 

From Theorem l3.1l we can easily derive examples of pairs (x, y) that are independent and 
which have constructive Hausdorff dimension e, for every rational e > 0. For example, if we 
start with x and y that are random with respect to each other and build x' = x(l) 0x(2) . . . 
(i.e., we insert Os in the even positions) and similarly build y' from y, then x' and y' have 
constructive Hausdorff dimension equal to 1/2 and are independent (because C x (y'\n) and 
C x {y \(n/2)) are within a constant of each other, as are C{y'\n) and C{y \{n/2))). The pairs 
of sequences from Theorem 13.11 (plus those derived from there as above) and those from 
Remark [5] are the only examples of independent sequences that we know. Thus, currently, 
we have examples of independent pairs (x, y) only for the case when x has maximal prefix- 
free complexity (i.e., x is random) or x is obtained via a straightforward transformation as 
above from a random sequence, and for the case when x has minimal prefix-free complexity 
(i.e., x is .ff-trivial) . We believe that for every x, there are sequences y independent with 
it, and moreover we believe that the set of sequences independent with x has measure one. 
For finitary-independence these facts are true. 

Theorem 3.2 Let x be an arbitrary sequence and let y be a sequence that is random 
conditioned by x. Then x and y are finitary-independent. 

Proof. Suppose x and y are not finitary-independent. Then there are infinitely many n 
with C(y\n \ x\n) < C(y\n) — 51ogn. Consider a constant c\ satisfying C(y\n) < n + c\, 
for all n. We get (under our assumption) that, for infinitely many n. C(y\n \ x\n) < 
n — 51ogn + ci. Then, by inequality for infinitely many n, C x ^ n (y\n) < n — 31ogn + c+ci. 
Note that that for every n and every m > n, C x ^ m {y\n) < C x ^ n {y\n). Thus, for infinitely 
many n and for all m > n, 

C xlm (y\n) <n-31ogn+(c + ci). (6) 
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On the other hand, y is random conditioned by x. Therefore, for all n, H x (y \n) > n — 0(1). 
Let U' be the universal machine underlying the complexity H(-) and let p* be the shortest 
program such that U' x (p*) = y\n (if there are ties, take p* to be the lexicographically 
smallest among the tying programs). Let m(n) = min(n, use(U' x (p*))) • Note that, for all n, 
H x (y\n) = H x ^ n \y\n). It follows that, for every n, H x ^ n \y\n) = H x {y\n) > n-O(l). 
Recall that for every strings u and v, C v (u) > H v {u) — 2 log \u\ — 0(1). Thus, for every n, 

C x ^ n \y\n)>n-2\ogn-0{\). (7) 

Inequalities (jBJ) and (JJJ) are contradictory. □ 

Theorem 3.3 For every x, the set {y \ y finitary-independent with x} has measure one. 

Proof. By the previous result, the set in the statement of the theorem contains the set 
{y | y random conditioned by x} which has measure one. □ 

Thus there are many (in the measure-theoretical sense) pairs of sequences that are 
finitary-independent. But is it possible to have such pairs satisfying a given constraint? 
We consider one instance of this general issue. 

Proposition 3.4 If x is a random sequence then there are y and z such that (y, z) are 
finitary-independent and x = y XOR z. 

Proof. Take a sequence y finitary-independent with x. Then, by Proposition 12.71 y and 
[x XOR y) are finitary-independent. By taking z = x XOR y, it follows that x = y XOR z, 
with y and z finitary-independent. □ 



4 Effective constructions of finitary-independent sequences 

The examples of (finitary-) independent sequences that we have provided so far are exis- 
tential (i.e., non-constructive). In this section we investigate to what extent it is possible 
to effectively construct such sequences. We show some impossibility results and there- 
fore we focus on the weaker type of independence, finitary-independence (clearly, if it is 
not possible to produce a pair of sequences that are finitary-independent, then it is also 
not possible to produce a pair of sequences that are independent). Since a C-logarithmic 
sequence is finitary-independent with any other sequence, the issue of constructibility is in- 
teresting if we also require that the sequences have complexity above that of C-logarithmic 
sequences (see Remark [5]). Such sequences are of course non-computable, and therefore 
the whole issue of constructibility appears to be a moot point. However this is not so if we 
assume that we already have in hand one (or several) non-computable sequence(s), and we 
want to build additional sequences that are finitary-independent. Informally speaking, we 
investigate the following questions: 
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Question (a) Is it possible to effectively construct from a sequence x another sequence 
y (unitary-) independent with x, where the independence is not trivial (recall Remark^? 
This question has two variants depending on whether we seek a uniform procedure (i.e., 
one procedure that works for all x), or whether we allow the procedure to depend on x. 

Question (b) Is it possible to effectively construct from a sequence x two sequences 
y and z that are (finitary-) independent, where the independence is not trivial? Again, 
there are uniform and non-uniform variants of this question. 

We analyze these questions in Section 14. 1[ Similar questions for the case when the 
input consists of two sequences x\ and X2 are tackled in Section f4.2l 

4.1 If we have one source 

We first consider the uniform variant of Question (a): Is there a Turing reduction / such 
that for all x £ {0,1}*, (x,f(x)) are (finitary-) independent? We even relax the require- 
ment and demand that / should achieve this objective only if x has positive constructive 
Hausdorff dimension (this only makes the following impossibility results stronger). 

As discussed above, we first eliminate some trivial instances of this question. Without 
any requirement on the algorithmic complexity of the desired f(x), the answer is trivially 
YES because we can take f{x) = W (or any other computable sequence). Even if we only 
require that f(x) is not computable, then the answer is still trivially YES because we can 
make f(x) to be C-logarithmic. For example, consider 

f(x) = x(l) x(2)0 x(3)000 . . . x(k) 0_^_0 .... 

2 fc -!-l 

Then f(x) is C-logarithmic, but not computable provided x is not computable, and (x, f(x)) 
are finitary-independent simply because f(x) is C-logarithmic. 

As noted above, the question is interesting if we require f(x) to have some "significant" 
amount of randomness whenever x has some "significant" amount of randomness. We 
expect that in this case the answer should be negative, because, intuitively, one should not 
be able to produce independence (this is property P3 in Remark [6]). 

We consider two situations depending on two different meanings of the concept of 
"significant" amount of randomness. 

Case 1: We require that f(x) is not C-logarithmic. We do not solve the question, but 
we show that every reduction / that potentially does the job must have non-polynomial 
use. 

Proposition 4.1 Let f be a Turing reduction. For every sequence x, if the function 
use°j-(n) is polynomially bounded, then x and f(x) are not finitary-independent, unless 
one of them is C-logarithmic. 
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Proof. Let y be f{x). Then for every n, let m(n) = maxfc< n usej(l n )). Then y\n 
depends only on x\m{n) and m(n) is polynomial in n. Then C[y\n \ x\m{n)) < O(logn). 
If x and y were Unitary- independent, then C{y\n) < C(y\(n) \ x\m(n)) + 0(logn + 
log m{n)) < 0(log(n)) + log(m(n)) < O(logn), for all n, i.e., y would be C-logarithmic . □ 

Case 2: We require that f(x) has complexity just above that of C-logarithmic se- 
quences (in the sense below). We show that in this case, the answer to the uniform variant 
of Question (a) is negative: there is no such /. The following definition introduces a class 
of sequences having complexity just above that of C-logarithmic sequences. 

Definition 4.2 A sequence x is C-superlogarithmic if for every constant c > 0, C{x\n) > 
clogra, for almost every n. 

The next proofs use the following facts. 

Fact 4.3 (Variant of Theorem 3.1 in [NR06J) For all rationals < a < (3 < 1, and for 
every set S that is infinite and computable, there exists a sequence x such that dim(rc) = a 
and for all wtt-reductions f, either f(x) does not exist or C(f(x)\n) < /3n, for infinitely 
many n in S. 

Fact 4.4 (Variant of Theorem 3.1 in [BDS07J) For every Turing reduction h, for all ratio- 
nals < a < (3 < 1, and for every set S that is infinite and computable, there is a sequence 
x with dim(x) > a such that either h{x) does not exist or C{h(x)\n) < (3n, for infinitely 
many n in S. 

Fact 4.5 (Theorem 4.15 in QZim07j) For any 5 > 0, there exist a constant c, a set S that 
is infinite and computable, and a truth-table reduction g : {0, 1}°° x {0, 1}°° — > {0, 1}°° 
(i.e., g is a Turing machine with two oracles) with the following property: 

If the input sequences x andy are finitary-independent and satisfy C{x\n) > cdogn and 
C{y\n) > cTogn, for almost every n, then the output z = f(x,y) satisfies C(f(x,y)\n) > 
(1 — 5) ■ n, for almost every n in S. 

Theorem 3.1 in |NR06| is for S = N (and is stronger in that a = j3) but its proof can be 
modified in a straightforward manner to yield Fact 14.31 Theorem 3.1 in I>I)S()7 is also for 
S = N and can also be modified in a simple manner - using Fact 14.31 - to yield Fact 14.41 

We can now state the impossibility results related to Case 2. To simplify the structure 
of quantifiers in the statement of the following result, we posit here the following task for 
a function / mapping sequences to sequences: 

TASK A: for every x £ {0, 1}°° with dim(x) > 0, the following should hold: 

(a) f(x) exists. 

(b) f(x) is C-superlogarithmic. 

(c) x and f{x) are finitary-independent. 
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Theorem 4.6 There is no Turing reduction f that satisfies TASK A. 

Proof. Suppose there exists / satisfying (a), (b) and (c) in TASK A. Let S be the 
infinite, computable set and let g be the truth-table reduction promised by Fact 14.51 for 
S = 0.3. Let h be the Turing reduction h(x) = g(x, /(x)). Let x* be the sequence promised 
by Fact 14.41 for a = 0.5, (3 = 0.6, and the above set S and Turing reduction h. On one 
hand, by Fact 14.41 C(h(x*) \n) < 0.6n, for infinitely many n£S, On the other hand, by 
Fact 14.51 C(h(x*)\n) > 0.7n, for almost every n G S. We have reached a contradiction. □ 

We next consider the uniform variant of Question (b). 

First we remark, that by van Lambalgen Theorem [vL90], if the sequence x is random, 
then x even and x oc id are random relative to each other (where x dd is x(l)x(3)x(5) . . . and 
x even is x(2)x(4)x(6) . . .). Thus, x even and x dd are certainly independent. 

Kautz |Kau03| has shown a much more general result by examining the splittings of 
sequences obtained using bounded Kolmogorov-Loveland selection rulescl He showed that 
if x is a random sequence, xo is the subsequence of x obtained by concatenating the bits of 
x chosen by an arbitrary bounded Kolmogorov-Loveland selection rule, and x\ consists of 
the bits of x that were not selected by the selection rule, then xq and x\ are random with 
respect to each other (and thus independent). 

We show that the similar result for sequences with constructive Hausdorff dimension 
a € (0, 1) is not valid. In fact, our next result is stronger, and essentially gives a negative 
answer to the uniform variant of Question (b). 

We posit the following task for two functions f\ and fi mapping sequences to sequences: 

TASK B: for every x € {0, 1}°° with dim(x) > 0, the following should hold: 

(a) /i(x) and /^(x) exist, 

(b) /i(x) and fi(x) are C-super logarithmic, 

(c) fi(x) and f2(x) are finitary-independent. 

Theorem 4.7 There are no Turing reductions fx and /2 satisfying TASK B. 

Proof Similar to the proof of Theorem 14.61 □ 

The non-uniform variants of Questions (a) and (b) remain open. In the particular 
case when / is a wtt-reduction, we present impossibility results analogous to those in 
Theorem 14.61 and Theorem 14.71 The proofs are similar, with the difference that we use 
Fact EO] instead of Fact IPl 

1 A Kolmogorov-Loveland selection rule is an effective process for selecting bits from a sequence. Infor- 
mally, it is an iterative process and at each step, based on the bits that have been already read, a new bit 
from the sequence is chosen to be read and (before that bit is actually read) the decision on whether that 
bit is selected or not is taken. A bounded Kolmogorov-Loveland selection rule satisfies a certain requirement 
of monotonocity for deciding the selected bits, see |Kau03| . 
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Theorem 4.8 For all rational a € (0,1), there exists dim(x) = a such that for every 
wtt-reduction f , at least one of the following holds true: 

(a) f(x) does not exist, 

(b) f(x) is not finitary-independent with x, 

(c) f(x) is not C-superlogarithmic. 

Theorem 4.9 For all rational a € (0, 1), there exists x with dim(x) = a such that for 
every wtt-reductions f\ and $2, at least one of the following holds true: 

(a) fi(x) does not exist or f2(x) does not exist, 

(b) f\{x) and /^(x) are not finitary-independent, 

(c) f\{x) is not C-superlogarithmic or f2(x) is not C-superlogarithmic. 

Theorem l4.9l has an interesting implication regarding sequences with constructive Hausdorff 
dimension in the interval (0, 1). Suppose, for example that we want to construct a sequence 
with constructive Hausdorff dimension 1/2. The first idea that comes to mind is to take a 
random sequence x = x(l)x(2) . . . and either consider the sequence y = x(l)0x(2)0 . . . (we 
insert 0s in all even positions) or the sequence z = x{\)x{\)x{2)x{2) . . . (we double every 
bit). The sequences y and z have constructive Hausdorff dimension 1/2. Theorem 14.91 
shows, roughly speaking, that there are sequences with dimension strictly between and 
1, where the partial randomness is due necessarily to one of the two methods stated above. 
Formally, for every rational a € (0, 1), there is a sequence x with dim(x) = a so that 
no matter what wtt method we use for selecting from x two subsequences, either one of 
the resulting subsequences has low complexity or the two resulting subsequences are not 
independent. 

4.2 If we have two sources 

We have seen some limits on the possibility of constructing a finitary-independent sequences 
starting from one sequence. What if we are given two finitary-independent sequences: is it 
possible to construct from them more finitary-independent sequences? 

First we observe that if x and y are two independent sequences and g is an arbitrary 
Turing reduction, then it does not necessarily follow that x and g{y) are independent (as 
one may expect). On the other hand it does follow that x and g(y) are finitary-independent. 

Proposition 4.10 [Ste07] There are two independent sequences x and y and a Turing 
reduction g such that x and g{y) are not independent. 

Proof. Let z be a random sequence and let u, v, and w be sequences such that z = u(Bv(Bw. 
By van Lambalgen Theorem |vL90| . each of the sequences u, v, and w are random relative 
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to the join of the other two. We define the sequences x and y as follows: 
x(2 n ) = u(n), for all n € N 

x(m) = v(m), for every m that is not a power of 2 
y(2 n ) = u(n), for all n € N 

y(m) = w(m), for every m that is not a power of 2 



Claim 4.11 The sequences x and y are independent. 

Proof. Suppose x and y are not independent. Then, similarly to Proposition 12.41 (e), for 
infinitely many n, C x {y\n) < C{y\n) — 7 logn. Then 

C u ® v {w\n) < C u ® v {y\n) + 2 log n + 0(l) 

(because w\n and y\n differ in only logn bits) 

< C x (y\n) + 2 logn + 0(1) 

(because queries to x can be replaced by queries to u and v) 

< C(y\n) -71ogn + 21ogn + 0(l), 

for infinitely many n 

< C(w \n) + 2 log n - 7 log n + 2 log n + 0(1) 
= 0(10 fn)- 2 log n + 0(l) 

< n-31ogn + 0(l). 

This contradicts that w is random with respect to u © v. □ 

It is easy to define a Turing reduction g such that g{y) = u. Notice that C x {u\n) = 
O(logn), because u is many-one reducible to x. On the other hand C{u\n) > n — 21ogn + 
O(l), for every n, because u is random. Therefore x and g(y) are not independent. □ 

We do not know if the facts that x and y are finitary-independent and g is a Turing 
reduction, imply that x and g(y) are finitary-independent This would show that finitary- 
dependency cannot be created. 

The following weaker result holds. 

Proposition 4.12 If x and y are independent, and g is a Turing reduction, then x and 
g{y) are finitary-independent (provided g(y) exists). 

Proof. Since x and y are independent, there exists a constant c such that for all n, 

C y (x\n) > C(x\n) — clogn. 

Suppose that x and g(y) are not finitary-independent. Then there are infinitely many n 
such that C{x\n \ g{y)\n) < C{x\n) — (c + 4)logn. Since C y (x\n) < C{x\n \ g{y)\n) + 
2 logn + 0(1), it would follow that, for infinitely many n, 

C y (x \n) < C{x \n) - (c + 1) log n, 

which contradicts the first inequality. □ 
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Corollary 4.13 There are sequences that are finitary-independent but not independent. 



Proof. The sequences x and g(y) from Proposition 14. 101 are not independent, but they are 
finitary-independent by Proposition 14. 12l □ 

As we mentioned, we do not know if Proposition 14.121 can be strengthened to hold if x 
and y are finitary-independent. However, for such x and y, there exists a simple procedure 
that starting with the pair (x,y), produces a new pair of finitary-independent sequences. 
Namely, we take the pair (x,y dd)- 

Proposition 4.14 If x and y are finitary-independent, then x and y &i are finitary- 
independent. 

Proof. Suppose that for every constant c there are infinitely many n such that C(x\n | 
Uodd\n) < C{x\n) — c-logn. Note that, for all n, C(x\n \ y\2n) < C(x\n \ y ot id\n) + 0(1). 
Our assumption implies that for every constant c there are infinitely many n such that 
C{x\n | y\2n) < C(x\n) — clogn + 0(1). By Proposition 12.41 (a), this contradicts the fact 
that x and y are finitary-independent. □ 

The next issue that we study is whether given a pair of (finitary-)independent strings 
(x, y), it is possible to effectively produce another string that is (finitary-)independent with 
both x and y. We give a positive answer for the case when x and y are both random. The 
similar question for non-random x and y remains open (but see Section 14. 3p . 

Theorem 4.15 There exists an effective transformation f with polynomially-bounded use 
such that if x and y are random and independent (respectively finitary-independent), then 
f(x,y) is independent (respectively, finitary-independent) with both x and y, and the inde- 
pendence is not trivial (recall Remark^). 

Remark: Contrast with Proposition 14. 11 where we have shown that for every x, for ev- 
ery effective transformation / with polynomially-bounded use, x and f(x) are not finitary- 
independent. 

Proof. We take f(x,y) = x XOR y and take into account Proposition 12.71 □ 
4.3 Producing independence: the finite case 

An interesting issue is whether given as input several sequences that are (finitary-) inde- 
pendent, there is an effective way to construct a sequence that is (finitary-) independent 
with each sequence in the input (and the independence is not trivial). A result of this 
type is obtained for the case when the input consists of two random sequences x and y in 
Theorem 14.151 We do not know if in Theorem 14.151 we can remove the assumption that x 
and y are random. 
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In what follows we will consider the simpler case of strings. In this setting we are able 
to give a positive answer for the situation when we start with threei input strings that are 
independent (and not necessarily random). First we define the analogue of independence 
for strings. 

Definition 4.16 Let c E M + and fcGN. We say that strings x\,x 2 , ■ ■ ■ ,Xk in {0, 1}* are 

c-independent if 

C{x\x 2 ...x k ) > C(xi) + C(x 2 ) + . . . + C(x fc ) - c(log \x\\ + log \x 2 \ + . . . + log \x k \). 

The main result of this section is the following theorem, whose proof draws from the 
techniques of |Zim07| . 

Theorem 4.17 For all constants a > and o\ € (0,a), there exists a computable function 
f : {0, 1}* x {0, 1}* x {0, 1}* — > {0, 1}* with the following property: For every c£l + there 
exists c € M + such that if the input consists of a triplet of c-independent strings having 
sufficiently large length n and plain complexity at least a-n, then the output is c-independent 
with each element in the input triplet and has length \_o~\n\ . 
More precisely, if 

(i) (x, y, z) are c-independent, 

(ii) \x\ = \y\ = \z\ = n, and 

(hi) C(x) > a ■ n, C(y) > a • n, C(z) > a ■ n, 

then, provided n is large enough, the following pairs of strings (f(x,y,z),x), (f(x,y,z),y), 
(f(x,y,z),z) are c-independent, \f(x,y,z)\ = \crin\, and C(f(x,y, z)) > \_a\n\ — O(logn). 

Before we delve into the proof, we establish several preliminary facts. 

Lemma 4.18 If x±, x 2 , X3 are three strings that are c-independent, then 

C{x\ I x 2 x 3 ) > C(x\) - (c + 2) (log \x\ \ + log |ic 2 | + log |x 3 |) - 0(1). 

Proof. The following inequalities hold for every three strings and in particular for the 
strings x\, x 2 , and X3: 

C(xix 2 x 3 ) < C(x 2 x 3 ) + C(x\ I x 2 x 3 ) + 21og |xi| + 0(1), 

and 

C(x 2 x 3 ) < C{x 2 ) + C{x 3 ) + 2 log \x 2 \ + O(l). 

2 The case when the input consists of two independent strings remains open. 
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Then 



C(xi | x 2 xz) > (7(21X2X3) - (7(2223) - 2 log \xi\ - 0(1) 

> C(xi) + C{x 2 ) + (7(2 3 ) - c(log \xi\ + log \x 2 \ + log \x 3 \) 
- (C(x 2 ) + (7(2 3 ) + 2 log |2 2 | + 0(1)) - 2 log |2i| - 0(1) 

> 0(21) - (c + 2)(log |xi| + log |x 2 | + log |2 3 j) - 0(1). 

□ 

The next lemma establishes a combinatorial fact about the possibility of coloring the 
cube [N] x [N] x [N] with M colors such that every planar rectangle contains all the 
colors in about the same proportion. Here N and M are natural numbers, [N] denotes 
the set {1,2,..., N}, [M] denotes the set {1,2,..., M}and a planar rectangle is a subset 
of [N] x [N] x [N] having one of the following three forms: B\ x B 2 x {k}, B\ x {k} x B 2 , 
or {k} x Si x B 2 , where k G [N], B\ C [N] and 5 2 C [iV]. 

Lemma 4.19 Let < o\ < a 2 < 1. For every n sufficiently large, it is possible to color 
the cube [2 n ] x [2 n ] x [2 n ] wit/i M = 2L CTin J colors in such a way that every planar rectangle 
satisfying ||-Bi|| = a2^ CT2n l and \\B 2 \\ = b2^ a ' 2n ^ for some natural numbers a and b contains 
at most (2/M)||i?i||||i?2|| occurrences of color c, for every color c G [M]. 

Proof. We use the probabilistic method. Let N = 2 n . We color each cell of the 
[N] x [N] x [N] cube with one color chosen independently and uniformly at random from 
[M\. For i,j,k G [N], let T(i,j,k) be the random variable that designates the color of 
the cell (i,j,k) in the cube. For every fixed cell (i,j,k) and for every fixed color c G [M], 
Prob(T(i, j, k) = c) = 1/M, because the colors are assigned independently and uniformly 
at random. Let us first consider some fixed subsets B\ and B 2 of [N] having size 2^1, a 
fixed k G [iV], and a fixed color c G [M]. Let A be the event "the fraction of occurences of 
c in the planar rectangle B\ x B 2 x {&;} is greater than 2/M." Using the Chernoff bounds, 
it follows that 

Prob(^)<e-( 1 /3)(i/M)7V^_ 

The same upper bounds hold for the probabilities of the similar events regarding the planar 
rectangles B\ x {k} x B 2 and {k} x B\ x B 2 . Thus, if we consider the event B "there is 
some color with a fraction of appearances in one of the three planar rectangles mentioned 
above greater than (2/M)", then, by the union bound, 

Prob(i?) OMe-^Xi/W^ (8) 

The number of ways to choose B 1 C [TV] with ||5i|| = 2^ 27l \ B 2 C [N] with ||S 2 || = 2^ 2n l 
and k G [N] is approximately (ignoring the truncation) (^ 2 ) • (^ 2 ) ' ^> which is bounded 
by 

2JV CT 2 _ 2AT CT 2 (l_ CT2 )ln(JV) . In AT (q) 
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(we have used the inequality (?) < (en/k) k ). Clearly, for our choice of M, the right hand 
side in Q times the right hand side in ([SD is less than 1. It means that there exists a 
coloring where no color appears a fraction larger than (2/M) in every planar rectangle 
with B\ and B 2 having size exactly 2^ rT2n \ For planar rectangles having the sizes of B\ 
and B2 an integer multiple of 2^ a2n \ the assertion holds as well because such rectangles 
can be partitioned into subrectangles having the size exactly 2^°~ 2n \ □ 

Proof of Theorem 14.171 We take n sufficiently large so that all the following inequal- 
ities hold. Let x* , y* and z* be a triplet of strings of length n satisfying the assumptions 
in the statement. Let N = 2 n and let us consider a constant 02 € (ax, a). By exhaustive 
search we find a coloring T : [N] x [N] x [N] — > [M] satisfying the properties in Lemma [4. 191 
Identifying the strings x*, y* and z* with their indeces in the lexicographical ordering of 
X n , we define w* = T(x* ,y* , z*). Note that the length of w* is logM = |_cri7T,J , which 
we denote m. We will show that C(w* \ z*) > m — c'logm, for d = 3c + d + 13, for a 
constant d that will be specified later. Since C(w*) < m + O(l), it follows that w* and z* 
are independent. In a similar way, it can be shown that w* and x* are independent, and 
w* and y* are independent. 

For the sake of obtaining a contradiction, suppose that C(w* \ z*) < m — d logm. The 
set A = {w I C(w I z*) < m — c'logm} has size < 2 m ~ c lo s m and, by our assumption, 
contains w* . 

Let ti be such that C(x*) = t% and t2 be such that C(y* \ z*) = £2- Note that t\ > a"2ra. 
The integer ti is also larger than a2fi, because C(y* \ z*) > C(y* \ z*x*) — 21ogn — O(l) > 
C{y*) — (c+4)(31ogn) — O(l) > an— (3c+12)logn — 0(1) > 02"- For the second inequality 
we have used Lemma 14.181 

Let Bi = {x e S n I C(x) < ti}. Note that B x has size bounded by 2* 1+1 . We take a 
set B[ including B\ having size exactly 2* 1+1 . Similarly, let B2 = {y € S n | C(y | z*) < ^2} 
and let B' 2 be a set that includes B2 and has size exactly 2* 2+1 . Let be the index of z* 
in the lexicographical ordering of S n . By Lemma 14.191 it follows that for every a £ [M] , 

HT-^a) n (B[ xB' 2 x {k})\\ < (2/M)\\B[\\\\B' 2 \\. 

Consequently, 

\\T-\A) n [B x x8 2 x {jfe})|| < \\T-\A) n (Sj x S 2 x {Jfe})|| 

= Ea e All^ 1 («)n( J B 1 x J B 2 x{fc})|| 

^ 2 m ~ c ' lo s" 1 . (2/2 m ) \\B'i || ||i?2 II = 2* 1+ * 2+3_c ' lo s m 

Note that given 2*, m — c'logm, ii and ti, we can enumerate T _1 (^4) n {B\ x B2 x {k}). 
Since (x* ,y* , z*) is in this set, it follows that the complexity of x*y* given z* is bounded 
by the rank of the triplet (x* ,y*,z*) in a fixed enumeration of the set and the information 
needed to perform the enumeration. Thus, 

C(x*y* \z*) < ti + t 2 + 3 - c' log m + 2 log(m - d log m) + 2 log t x + 2 log t 2 + 0(1) 
< h + t 2 - (d - 2) logm + 21ogti + 21ogt 2 + 0(1). 
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On the other hand, by the conditional version of the Symmetry of Information Equa- 
tion ([3]), there exists a constant d such that for all strings u,v,w, C(uv \ w) > C(v \ 
w) + C(u \ uw) — d(log \uv\). It follows that 

C{x*y* | z*) > C(y* \ z*) + C{x* \ y*z*) - dlogn- 0(1) 
>t 2 + h - (c + 2)(31ogn) -dlogn-O(l) 
= h + t 2 - (3c + d + 6) log n - O(l). 

For the second inequality we have used Lemma 14.181 Note that t\ < n + 0(1) and t 2 < 
n + O(l) and m = o\n. Combining the above inequalities, we obtain (c' — 2)logo"in < 
(3c + d + 10) log n + 0(1). Since c' = 3c + d + 13, we have obtained a contradiction. □ 
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